Method and system for evaluating and monitoring compliance using emotion detection

ABSTRACT

A system for administering, evaluating, and monitoring a subject&#39;s compliance with task performance requirements within an action programme comprising optical sensors for capturing subject&#39;s facial expression, eye movements, point-of-gaze, and head pose of a subject during a compliance evaluation and monitoring session; a domain knowledge data repository comprising concept data entities, each having knowledge and skill content items, and task data entities, each having lecture content material items; a subject module configured to estimate the subject&#39;s affective state and cognitive state using the sensory data collected from the optical sensors; and a trainer module configured to select a task data entity for delivery and presentment to the subject after each completion of a task data entity based on a probability of the subject&#39;s understanding of the associated concept data entity&#39;s knowledge and skill content items and a probability of the subject achieving a target compliance level.

CROSS-REFERENCES WITH RELATED DOCUMENTS

This application is a national phase entry of International PatentApplication no. PCT/IB2018/053968 filed on Jun. 4, 2018 which claimspriority to U.S. Patent Application No. 62/520,542 filed 15 Jun. 2017,U.S. patent application Ser. No. 15/647,272 filed 12 Jul. 2017, U.S.Patent Application No. 62/594,557 filed 5 Dec. 2017, U.S. PatentApplication No. 62/622,888 filed 27 Jan. 2018, U.S. Patent ApplicationNo. 62/627,734 filed 7 Feb. 2018, and U.S. Patent Application No.62/646,365 filed 21 Mar. 2018; the disclosures of which are incorporatedherein by reference in their entirety.

FIELD OF THE INVENTION

The present invention relates generally to methods and systems forproviding and delivery of compliance administration and monitoring inthe contexts of educational programmes and training, including corporatetraining, academic tutoring, in-class and out-of-class learnings,medical treatment and health improvement programmes, sport training,fitness and lifestyle programmes, correctional service andrehabilitation programmes, as well as governmental law and regulatoryenforcement, and behavioral standards for individuals. Particularly, thepresent invention relates to methods and systems for administering,evaluating and monitoring compliance with task performance requirementsin an action programme.

BACKGROUND OF THE INVENTION

Compliance means conforming to certain task performance specificationsas required by an action programme by a subject enrolled therein.Conventional compliance evaluation and monitoring techniques focus onproviding one-time testing with clearly defined passing criteria tosubjects in an action programme. Many of these conventional techniquesfocus only on obtaining pass/fail indicators as effectively as possiblewithout any capability for predicting the subject's progress or guidingthe subject towards total compliance. Further, these traditionalcompliance evaluation and monitoring methods often rely on one-time orsparse, at best, manual administrations of tests. This can hardlyprovide assurance for compliance on a continuous basis.

To address aforementioned issues, it would be desirable to have anintelligent compliance evaluation and monitoring system that models theaffective and cognitive states of the subject, assist the complianceofficer in providing personalized compliance instructions andquestionnaire in guiding the subject toward total compliance,continuously monitoring the subject's task performance and behavior, andminimize overhead activities associated with the compliance programme.

SUMMARY OF THE INVENTION

The present invention provides a method and system for administering,evaluating, and monitoring a subject's compliance with task performancerequirements within an action programme using one or more of sensing ofthe subject's gestures, emotions, and movements, speech and voicerecognition, behavior pattern recognition, and quantitative measurementsof questionnaire results and task performances. In accordance to variousembodiments of the present invention, the subject within the actionprogramme is to perform certain tasks according to performancespecifications. In accordance to one embodiment, the method and systemevaluate and monitor the compliance of the subject by periodicallyadministering one or more questionnaires and analyzing the subject'sresponses to the questionnaires; and by continuously monitoring thesubject's performance of one or more tasks under performancespecification requirements.

In accordance to one aspect of the present invention, the method andsystem estimates the affective state and cognitive state of the subjectby image and/or video capturing and analyzing the subject's facialexpression, eye movements, point-of-gaze, and head pose; and physiologicdetection, such as tactile pressure exerted on a tactile sensing device,subject's handwriting, tone of voice, and speech clarity during thesubject is responding to the questionnaire or during a sampling timewindow when the subject is performing the task procedure. The image orvideo capture can be accomplished by using built-in or peripheralcameras in desktop computers, laptop computers, tablet computers, and/orsmartphones used by the subject in responding to the questionnaire,and/or other optical sensing devices placed and installed in theenvironments within which the subject performs the tasks in the actionprogramme. The captured images and/or videos are then analyzed usingmachine vision techniques. For example, stalled eye movements,out-of-focus point-of-gaze, and a tilted head pose are signalsindicating lack of interest and attention toward, and/or lack ofknowledge in the subject matters being presented in the questionnaireand task procedural instructions, untruthfulness in answering thequestionnaire, or lack of skill/knowledge in the tasks at hands; while astrong tactile pressure detected is a signal indicating anxiety, lack ofconfidence, and/or frustration in the subject matters being presented inthe questionnaire and task procedural instructions or of the tasks athands; either could represent a tendency of low level of compliance ornoncompliance.

In accordance to one embodiment, selected performance data andbehavioral data from the subject are also collected in determining thesubject's comprehension of and level of engagement in the materialspresented in the questionnaires and task procedural instructions fortasks required to be performed in the action programme. These selectedperformance data and behavioral data include, but not limited to,correctness of answers to questions in the questionnaire, number ofsuccessful and unsuccessful attempts, closeness of the subject's answersto model answers, number of toggling between given answer choices, andresponse speed to questions of certain types, and subject matters. Forexample, the subject's excessive toggling between given choices and slowresponse speed in answering a question indicate doubts and hesitationson the answer to the question.

The affective state and cognitive state estimation and performance dataare primarily used in gauging the subject's level of compliance withperformance specifications of tasks in an action programme. While asingle estimation is used in providing a snapshot assessment of thesubject's progress toward total compliance in her task performance andprediction of the subject's eventual achievable level of compliance,multiple estimations are used in providing an assessment history andtrends of the subject's progress. Furthermore, the estimated affectivestates and cognitive states of the subject are used in the modeling ofthe compliance programme in terms of choice of methods of complianceevaluation and monitoring, and instruction delivery and administration.

In accordance to another aspect of the present invention, the method andsystem provide a mechanism for delivering and managing interactive andadaptive compliance questionnaire and task procedural instructions. Themechanism logically structures the questionnaire and task proceduralinstructions materials and the delivery mechanism data for evaluatingand monitoring compliance in an action programme as Domain Knowledge,and its data are stored in a Domain Knowledge repository. A DomainKnowledge repository comprises one or more Concept objects and one ormore Task objects. Each Concept object comprises one or more Knowledgeand Skill items. The Knowledge and Skill items are ordered by taskperformance specification complexity/difficulty/stringency levels, andtwo or more Knowledge and Skill items can be linked to form aCurriculum. In the case where the present invention is applied in aparticular industry or business, a Curriculum defined by the presentinvention may be the equivalence of the operation manual/standard andthere is one-to-one relationship between a Knowledge and Skill item anda task performance specification in the operation manual/standard. TheConcept objects can be linked to form a logical tree data structure forused in a Task selection process.

Each Task object has various task procedural instruction materials. EachTask object is associated with one or more Concept objects in aCurriculum. In accordance to one embodiment, a Task object can beclassified as: Basic Task, Interactive Task, or Task with an UnderlyingCognitive or Expert Model. Each Basic Task comprises one or moreoperation notes, task procedural instructions, illustrations, testquestions and answers designed to assess whether the subject has readall the materials. Each Interactive Task with an Underlying Cognitive orExpert Model comprises one or more task procedures each comprises one ormore instructional steps designed to guide the subject in completing thetask procedure according to performance specification. Each stepprovides an answer, common misconceptions, and hints. The steps are inthe order designed to follow the delivery flow of a task procedure. Thisallows a tailored scaffolding (e.g. providing guidance and/or hints) foreach task based on a point in a task procedure executed.

In accordance to another aspect of the present invention, the mechanismfor delivering and managing interactive and adaptive compliancequestionnaires and instructions logically builds on top of the DomainKnowledge two models of operation: Subject Model and Training Model.Under the Subject Model, the system executes each of one or more of theTask objects associated with a Curriculum in a Domain Knowledge in awork session for a subject. During the execution of the Task objects,the system measures the subject's performance and obtain the subject'sperformance metrics in each Task such as: the numbers of successful andunsuccessful attempts to complete the instructional steps in the Task,number of hints requested, and the time spent in completing the Task.The performance metrics obtained, along with the information of the Taskobject, such as its specification complexity/difficulty/stringencylevel, are fed into a logistic regression mathematical model of eachConcept object associated with the Task object. This is also called theknowledge trace of the subject, which is the calculation of aprobability of the subject achieving a target compliance level in a taskassociated with the with the task performance specification in theConcept object. The advantages of the Subject Model include that theexecution of the Task objects can adapt to the changing ability of thesubject. For non-limiting example, following the Subject Model, thesystem can estimate the compliance level achievable by the subject,estimate how much performance improvement can be expected for a nextTask, and provide a prediction of the subject's level of compliance in afuture point of time. These data are then used in the Training Model andenable hypothesis testing to make further improvement to the system,evaluate compliance officer quality and compliance questionnaire andtask procedural instruction material quality.

Under the Training Model, the system receives the data collected fromthe execution of the Task objects under the Subject Model and the DomainKnowledge for making decisions on the instruction delivery strategy andproviding feedbacks to the subject and compliance officer. Under theTraining Model, the system is mainly responsible for executing thefollowings:

1.) Define the entry point for the first Task. Initially all indicatorsfor Knowledge and Skill items are set to defaults, which are inferredfrom data in either an application form filled by the subject orcompliance officer or an initial assessment of the subject by thecompliance officer. Select the sequence of Tasks to execute. To selectthe next Task, the system's trainer module has to search through alogical tree data structure of Concept objects, locate a Knowledge andSkill with the lowest skill level and then use a question matrix tolookup the corresponding Task items that match the learning traits ofthe subject. Once selected, the necessary compliance questionnaire andtask procedural instruction materials are pulled from the DomainKnowledge, and send to the system's communication module for deliverypresentation in the system's communication module user interface.2.) Provide feedback. While the subject is working on a Task objectbeing executed, the system's trainer module monitors the time spent oneach Task step. When a limit is exceeded, feedback is provided as afunction of the current affective state of the subject. For example,this can be an encouraging, empathetic, or challenging message selectedfrom a generic list, or it is a dedicated hint from the DomainKnowledge.3.) Drive the system's pedagogical agent. The system's trainer modulematches the current affective state of the subject with the availablestates in the pedagogical agent. Besides providing the affective stateinformation, text messages can be sent to the system's communicationmodule for rendering the pedagogical agent in a user interface.4.) Decide when a Concept is mastered. As described earlier, under theSubject Model, the system estimates the probability of the subjectachieving a target compliance level in a task associated with the taskperformance specification materials in each Concept. Based on apredetermined threshold (e.g. 95%), the compliance officer can decidewhen a Concept is mastered.

In accordance to another aspect of the present invention, the method andsystem for administering, evaluating, and monitoring a subject'scompliance with task performance requirements within an action programmeincorporate machine learning techniques that are based on interlinkingmodels of execution comprising: a Domain Model, an Assessment Model, aLearner Model, a Deep Learner Model, one or more Motivational Models, aTransition Model, and a Pedagogical Model. The interlinking models ofexecution is purposed for driving, inducing, or motivating certaindesirable actions, behavior, and/or outcome from the subject. Thesecertain desirable actions and/or outcome can be, as non-limitingexamples, mastering certain task procedures, adopting certain desirablebehaviors, achieving certain job assignment goals, making certainpurchases, and conducting certain commercial activities. Therefore,these interlinking models of execution are also applicable in the fieldsof corporate training and commercial retailing and trading.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of the invention are described in more detail hereinafterwith reference to the drawings, in which:

FIG. 1 depicts a schematic diagram of a system for administering,evaluating, and monitoring a subject's compliance with task performancerequirements within an action programme in accordance to one embodimentof the present invention;

FIG. 2 depicts a logical data flow diagram of the system;

FIG. 3 depicts an activity diagram of a method for administering,evaluating, and monitoring a subject's compliance with task performancerequirements within an action programme to one embodiment of the presentinvention;

FIG. 4 depicts a flow diagram of an iterative machine learning workflowused by the system in calculating a probability of achieving a targetcompliance level by the subject;

FIG. 5 illustrates a logical data structure used by the system inaccordance to one embodiment of the present invention;

FIG. 6A depicts a logical block diagram of interlinking models ofexecution in accordance to one aspect of the present invention; and

FIG. 6B shows the logical components details of the interlinking modelsof execution.

DETAILED DESCRIPTION

In the following description, methods and systems for administering,evaluating, and monitoring a subject's compliance with task performancerequirements within an action programme and the likes are set forth aspreferred examples. It will be apparent to those skilled in the art thatmodifications, including additions and/or substitutions may be madewithout departing from the scope and spirit of the invention. Specificdetails may be omitted so as not to obscure the invention; however, thedisclosure is written to enable one skilled in the art to practice theteachings herein without undue experimentation.

In accordance to various embodiments of the present invention, themethod and system for administering, evaluating, and monitoring asubject's compliance with task performance requirements within an actionprogramme use a combination of sensing of the subject's gestures,emotions, and movements, and quantitative measurements of questionnaireresults and task performances.

In accordance to one aspect of the present invention, the method andsystem estimate the affective state and cognitive state of the subjectby image and/or video capturing and analyzing the subject's facialexpression, eye movements, point-of-gaze, and head pose, and hapticfeedback, such as tactile pressure exerted on a tactile sensing,subject's handwriting, tone of voice, and speech clarity during thesubject is responding to the questionnaire or during a sampling timewindow when the subject is performing the task procedure. The image orvideo capture can be performed by using built-in or peripheral camerasin desktop computers, laptop computers, tablet computers, and/orsmartphones used by the subject, and/or other optical sensing devicesplaced and installed in the environments within which the subjectperforms the tasks in the action programme. The captured images and/orvideos are then analyzed using machine vision techniques. For example,stalled eye movements, out-of-focus point-of-gaze, and a tilted headpose are signals indicating lack of interest and attention, and/or lackof knowledge in the subject matters being presented in questionnaire,untruthfulness in answering the questionnaire, or lack ofskill/knowledge in the tasks at hands; while a strong tactile pressuredetected is a signal indicating anxiety, lack of confidence, and/orfrustration in the subject matters being presented in questionnaire orof the tasks at hands; either could represent a tendency of low level ofcompliance or noncompliance.

In accordance to one embodiment, selected performance data andbehavioral data from the subject are also collected in the affectivestate and cognitive state estimation. These selected performance dataand behavioral data include, but not limited to, number of successfuland unsuccessful attempts to task procedural step completions, speed incompleting task procedures, correctness of answers to questions in thequestionnaire, number of successful and unsuccessful attempts toquestions, closeness of the subject's answers to model answers, togglingbetween given answer choices, and response speed to test questions ofcertain types, subject matters, and/or task performance specificationcomplexity/difficulty/stringency levels, working steps toward asolution, the subject's handwriting, tone of voice, and speech clarity.For example, the subject's excessive toggling between given choices andslow response speed in answering a test question indicating doubts andhesitations on the answer to the question. The subject's intermediateworking steps toward completing a task procedural step are captured formatching with the model solution and in turn provides insight to thesubject's understanding of the task procedural instruction and taskperformance specification materials.

In accordance to various embodiments, the system for administering,evaluating, and monitoring a subject's compliance with task performancerequirements within an action programme comprises a sensor handlingmodule implemented by a combination of software and firmware executed ingeneral purposed and specially designed computer processors. The sensorhandling module manages the various sensors employed by the system. Thesensor handling module is in electrical and/or data communications withvarious electronic sensing devices including, but not limited to,optical and touch sensing devices; input devices including, but notlimited to, keyboard, mouse, pointing device, stylus, and electronicpen; image capturing devices; and cameras.

During the operation of the system, input sensory data are continuouslycollected at various sampling rates and averages of samples of inputsensory data are computed. In order to handle the different samplingrates of different sensing devices, a reference rate is chosen (e.g. 5Hz). A slower sampling rate input sensory data is interpolated with zeroorder hold and then sampled at the reference rate. A higher samplingrate input sensory data is subsampled at the reference rate. After thesample rate alignment, a trace of the last few seconds is kept in memoryafter which the average is calculated. Effectively this produces amoving average of an input sensory data and acts as a low-pass filter toremove noise.

Eye movements, Point-of-Gaze, and Head Pose Detection

In one embodiment, a low-cost optical sensor built-in in a computingdevice (e.g. subject facing camera in a tablet computer) is used. At arate of minimal 5 Hz, images are obtained from the sensor. Each image isthen processed by face/eye tracking and analysis systems known in theart. The three-dimensional (3D) head orientation is measured in Eulerangles (pitch, yaw, and roll). To measure the point-of-gaze, a 3D vectoris assumed from the origin of the optical sensor to the center of thepupil of the user, secondly, a 3D vector is determined from the centerof the eye-ball to the pupil. These two vectors are then used tocalculate the point of gaze. A calibration step helps to compensate foroffsets (subject position behind the screen, camera position relative tothe screen). Using this data, the planar coordinate of the gaze on thecomputer screen can be derived.

Facial Expressions and Emotions Determination

In another embodiment, the images and/or videos captured as mentionedabove, are processed to identify key landmarks on the face such as eyes,tip of the nose, corners of the mouth. The regions between theselandmarks are then analyzed and classified into facial expressions suchas: attention, brow furrow, brow raise, cheek raise, chin raise, dimpler(lip corners tightened and pulled inwards), eye closure, eye widen,inner brow raise, jaw drop, lid tighten, lip corner depression, lippress, lip pucker (pushed forward), lip stretch, lip such, mouth open,nose wrinkle, smile, smirk, upper lip raise. These expressions are thenmapped, using a lookup table, onto the following emotions: anger,contempt, disgust, engagement (expressiveness), fear, joy, sadness,surprise and valence (both positive as negative nature of the person'sexperience). Each emotion is encoded as a percentage and outputsimultaneously.

Physiologic Measurement

In another embodiment, the system comprises a wearable device to measurephysiologic parameters not limiting to: heart rate, electro dermalactivity (EDA) and skin temperature. This device is linked wirelessly tothe client computing device (e.g. tablet computer or laptop computer).The heart rate is derived from observations of the blood volume pulse.The EDA measures skin conductivity as an indicator for sympatheticnervous system arousal. Based on this, features related to stress,engagement, and excitement can be derived. Another approach is to usevision analysis techniques to directly measure the heart rate based onthe captured images. This method is based on small changes in lightabsorption by the veins in the face, when the amount of blood varies dueto the heart rate.

Handwriting Analysis

In another embodiment, test answers may be written on a dedicated notepaper using a digital pen and receive commands such as ‘task proceduralstep completed’. The written answer is then digitized on the fly and viaan intelligent optical character recognition engine, the system canevaluate the content written by the subject and provide any necessaryfeedback to guide the subject when needed. Studies show that takinglonghand notes encourages subjects to process and reframe information,improving the compliance results. Alternatively, embodiments may use OCRafter the tasks has been completed. The paper is scanned using a copierand the digitized image is fed to OCR software.

Speech and Voice Recognition and Analysis

In another embodiment, the system comprises one or more voice recordingdevices for recording the subject's speech during a complianceevaluation and monitoring session. The subject's speech is thendigitized on the fly and via an intelligent voice recognition engine,the system can evaluate the content spoken by the subject and provideany necessary feedback to guide the subject when needed. The substantivecontent of the subject's speech is recognized for verbal commandsrelated to a task procedure and/or verbal answers to questionnaire testquestions for further compliance analysis. The subject's voice andspeech clarity are recognized as input to the affective state andcognitive state estimation.

Pedagogical Agent-Subject Interaction

As a non-limiting example, a pedagogical agent may be non-human animatedcharacter with human traits implemented by a combination of softwareand/or firmware running in one or more general purposed computerprocessors and/or specially configured computer processors. It candisplay the basic emotions by selecting from a set of animations (e.g.animated GIFs), or by using scripted geometric transformation on astatic image displayed to the subject in a user interface. Anothermethod is to use SVG based animations. The animation can be annotatedwith text messages (e.g. displayed in a balloon next to the animation).The text messages are generated by and received from the trainer moduleof the system. The subject's responses to the pedagogical agent arereceived by the system for estimating the subject's affective state.

The affective state and cognitive state estimation and performance dataare primarily used in gauging the subject's level of compliance withperformance specifications of tasks in an action programme. While asingle estimation is used in providing a snapshot assessment of thesubject's progress toward total compliance in her task performance andprediction of the subject's eventual achievable level of compliance,multiple estimations are used in providing an assessment history andtrends of the subject's progress. Furthermore, the estimated affectivestates and cognitive states of the subject are used in the modeling ofthe compliance programme in terms of choice of methods of complianceevaluation and monitoring, and instruction delivery and administration.

Domain Knowledge

Referring to FIG. 5. In accordance to one aspect of the presentinvention, the method and system logically structure the compliancequestionnaire and task procedural instruction materials, and thedelivery mechanism in a compliance programme as Domain Knowledge 500. ADomain Knowledge 500 comprises one or more Concept objects 501 and oneor more Task objects 502. Each Concept object 501 comprises one or moreKnowledge and Skill items 503. The Knowledge and Skill items 503 areordered by task performance specificationcomplexity/difficulty/stringency levels, and two or more Concept objects501 can be grouped to form a Curriculum. In the case where the presentinvention is applied in a particular industry or business, a Curriculumdefined by the present invention may be the equivalence of the operationmanual/standard and there is one-to-one relationship between a Knowledgeand Skill item and a task performance specification in the operationmanual/standard. The Concept objects can be linked to form a logicaltree data structure (Knowledge Tree) such that Concept objects havingKnowledge and Skill items that are fundamental and/or basic in a topicare represented by nodes closer to the root of the logical tree andConcept objects having Knowledge and Skill items that are more advanceand branches of some common fundamental and/or basic Knowledge and Skillitems are represented by nodes higher up in different branches of thelogical tree.

Each Task object 502 has various compliance questionnaire and taskprocedural instruction content materials 504, and is associated with oneor more Concept objects 501 in a Curriculum. The associations arerecorded and can be looked up in a question matrix 505. In accordance toone embodiment, a Task object 502 can be classified as: Basic Task,Interactive Task, or Task with an Underlying Cognitive or Expert Model.Each Basic Task comprises one or more operation notes, task proceduralinstructions (e.g. video clips and other multi-media content), testquestions and answers designed to assess whether the subject has readall the materials. Each Interactive Task with an Underlying Cognitive orExpert Model comprises one or more problem-solving exercises eachcomprises one or more steps designed to guide the subject in derivingthe solutions to problems. Each step provides an answer, commonmisconceptions, and hints. The steps are in the order designed to followthe delivery flow of a task procedure. This allows a tailoredscaffolding (e.g. providing guidance and/or hints) for each task basedon a point in a task procedure executed.

In accordance to various embodiments, a Task object gathers a set ofcompliance questionnaire and task procedural instruction materials (e.g.operation notes and illustrations) relevant in the achievement of acompliance level. In addition to the aforementioned classification, aTask can be one of the following types:

1.) Reading Task: operation notes or illustrations to introduce a newtopic without grading, required to be completed before proceeding to aPractice Task is allowed;

2.) Practice Task: a set of questions from one topic to practice onquestions from a new topic until a threshold is reached (e.g. fiveconsecutive successful attempts without hints, or achieve anunderstanding level of 60% or more);

3.) Mastery Challenge Task: selected questions from multiple topics tolet the subject achieves mastery (achieve an understanding level of 95%or more) on a topic, and may include pauses to promote retention ofknowledge (e.g. review opportunities for the subjects); or4.) Group Task: a set of questions, problem sets, and/or problem-solvingexercises designed for peer challenges to facilitate more engagementfrom multiple subjects in a focus group, may be ungraded.

In accordance to one embodiment, the Domain Knowledge, its constituentTask objects and Concept objects, Knowledge and Skill items andCurriculums contained in each Concept object, operation notes,illustrations, test questions and answers in each Task object are dataentities stored a relational database accessible by the system (a DomainKnowledge repository). One or more of Domain Knowledge repositories mayreside in third-party systems accessible by the system foradministering, evaluating, and monitoring a subject's compliance withtask performance requirements within an action programme.

In accordance to another aspect of the present invention, the mechanismfor delivering and managing interactive and adaptive compliancequestionnaires and task procedural instructions logically builds on topof the Domain Knowledge two models of operation: Subject Model andTraining Model.

Subject Model

Under the Subject Model, the system executes each of one or more of theTask objects associated with a Curriculum in a Domain Knowledge for asubject. During the execution of the Task objects, the system measuresthe subject's performance and obtain the subject's performance metricsin each Task such as: the numbers of successful and unsuccessfulattempts to questions in the Task, number of hints requested, and thetime spent in completing the Task. The performance metrics obtained,along with the information of the Task object, such as its specificationcomplexity/difficulty/stringency level, are fed into a logisticregression mathematical model of each Concept object associated with theTask object. This is also called the knowledge trace of the subject,which is the calculation of the probability of the subject achieving atarget compliance level in a task associated with the Concept object. Inone embodiment, the calculation of a probability of achieving a targetcompliance level uses a time-based moving average of subject's answerscores to questions in the questionnaires with lesser weight on olderattempts, the number of successful attempts, number of failed attempts,success rate (successful attempts over total attempts), time spent, andtask performance specification complexity/difficulty/stringency level.In another embodiment, the calculation of a probability of achieving atarget compliance level uses a time-based moving average of subject'scompletion of task procedural steps with lesser weight on olderattempts, the number of successful attempts, number of failed attempts,success rate (successful attempts over total attempts), time spent, andtask performance specification complexity/difficulty/stringency level.

In one embodiment, the system calculates the probability of the subjectachieving a target compliance level in a task associated with the taskperformance specification in the Concept object using an iterativemachine learning workflow to fit mathematical models on to the collecteddata (subject's performance metrics and information of the Task)including, but not limited to, a time-based moving average of subject'sanswer scores to questions in the questionnaires with lesser weight onolder attempts, the number of successful attempts, number of failedattempts, success rate (successful attempts over total attempts), timespent, topic difficulty, and question difficulty. FIG. 4 depicts a flowdiagram of the aforesaid iterative machine learning workflow. In thisexemplary embodiment, data is collected (401), validated and cleansed(402); then the validated and cleansed data is used in attempting to fita mathematical model (403); the mathematical model is trainediteratively (404) in a loop until the validated and cleansed data fitthe mathematical model; then the mathematical model is deployed (405) toobtain the probability of the subject achieving a target compliancelevel in a task associated with the task performance specification inthe Concept object; the fitted mathematical model is also looped back toand used in the step of validating and cleansing of the collected data.

The knowledge trace of the subject is used by the system in driving Taskcompliance questionnaire and task procedural instruction material itemsselection, driving Task object selection, and driving compliancequestionnaire and task procedural instruction material ranking. Theadvantages of the Subject Model include that the execution of the Taskobjects can adapt to the changing ability of the subject. Fornon-limiting example, under the Subject Model the system can estimatethe compliance level achievable by the subject, estimate how muchperformance improvement can be expected for the next Task, and provide aprediction of the subject's level of compliance in a future point oftime. These data are then used in the Training Model and enablehypothesis testing to make further improvement to the system, evaluatecompliance officer quality and compliance questionnaire and taskprocedural instruction material quality.

Training Model

Under the Training Model, the system's trainer module receives the datacollected from the execution of the Task objects under the Subject Modeland the Domain Knowledge for making decisions on the compliancequestionnaire and task procedural instruction delivery strategy andproviding feedbacks to the subject and compliance officer. The systemfor administering, evaluating, and monitoring a subject's compliancewith task performance requirements within an action programme comprisesa trainer module implemented by a combination of software and firmwareexecuted in general purposed and specially designed computer processors.In one embodiment, the trainer module resides in one or more servercomputers. The trainer module is primarily responsible for executing themachine instructions corresponding to the carrying-out of the activitiesunder the Training Model. Under the Training Model, the trainer moduleexecutes the followings:

1.) Define the entry points for the Tasks execution. Initially allindicators for Concept Knowledge and Skill items are set to defaults,which are inferred from data in either an application form filled by thesubject or compliance officer or an initial assessment of the subject bythe compliance officer. Select the subsequent Task to execute. To selectthe next Task, the system's trainer module searches through a logicaltree data structure of Concept objects (Knowledge Tree), locate aConcept Knowledge and Skill with the lowest skill level (closest to theroot of the Knowledge Tree) and then use a matching matrix to lookup thecorresponding Task object for making the selection. Once selected, theTask object data is retrieved from the Domain Knowledge repository, andsend to the system's communication module for delivery presentation.2.) Provide feedback. While the subject is working on a Task objectbeing executed, the system's trainer module monitors the time spent on aTask step. When a time limit is exceeded, feedback is provided as afunction of the current affective state of the subject. For example,this can be an encouraging, empathetic, or challenging message selectedfrom a generic list, or it is a dedicated hint from the DomainKnowledge.3.) Drive the system's pedagogical agent. The system's trainer modulematches the current affective state of the subject with the availablestates in the pedagogical agent. Besides providing the affective stateinformation, text messages can be sent to the system's communicationmodule for rendering along with the pedagogical agent's action in a userinterface displayed to the subject.4.) Decide when a Concept is mastered. As described earlier, under theSubject Model, the system estimates the probability of the subjectachieving a target compliance level in a task associated with the taskperformance specification materials in each Concept. Based on apredetermined threshold (e.g. 95%), the compliance officer can decidewhen a Concept is mastered.

In accordance to various embodiments, the system for administering,evaluating, and monitoring a subject's compliance with task performancerequirements within an action programme further comprises acommunication module implemented by a combination of software andfirmware executed in general purposed and specially designed computerprocessors. In one embodiment, one part of the communication moduleresides and is executed in one or more server computers, and other partof the communication module resides and is executed in one or moreclient computers including, but not limited to, desktop computers,laptop computers, tablet computers, smartphones, and other mobilecomputing devices, among which some are dedicated for use by thesubjects and others by compliance officer.

The communication module comprises one or more user interfaces designedto present relevant data from the Domain Knowledge and materialsgenerated by the system operating under the Subject Model and TrainingModel to the subjects and the compliance officers. The user interfacesare further designed to facilitate user interactions in capturing userinput (textual, gesture, image, and video inputs) and displayingfeedback including textual hints and the simulated pedagogical agent'sactions. Another important feature of the communication module is toprovide an on-screen (the screen of the computing device used by asubject) planar coordinates and size of a visual cue or focal point forthe current Task object being executed. For a non-limiting example, whenan operation note from a Task object is being displayed on screen, thecommunication module provides the planar coordinates and size of theoperation note display area and this information is used to match withthe collected data from a point-of-gaze tracking sensor in order todetermine whether the subject is actually engaged in the Task (lookingat the operation note).

FIG. 2 depicts a logical data flow diagram of the system foradministering, evaluating, and monitoring a subject's compliance withtask performance requirements within an action programme in accordanceto various embodiments of the present invention. The logical data flowdiagram illustrates how the major components of the system work togetherin a feedback loop in the execution during the Subject Model andTraining Model. In an exemplary embodiment in reference to FIG. 2,during enrollment, a suitable series of tasks is selected by the subjectin an action programme. This series of tasks corresponds directly to aCurriculum object, which is a set of linked Concept objects in theDomain Knowledge 202, and constitutes the target compliance level 201for this subject. Upon the subject logging into the system via a userinterface rendered by the system's communication module, under theTraining Model, the system's trainer module selects and retrieves fromthe Domain Knowledge 202 a suitable Concept object and the associatedfirst Task object. Entering the Subject Model, the Task object data isretrieved from the Domain Knowledge repository, the system renders theTask object data (e.g. operation notes) on the user interface for thesubject, and the subject starts working on the task. Meanwhile, thesystem manages the compliance evaluation and monitoring process 203 bycollecting affective state sensory data including, but not limited to,point-of-gaze, emotion, and physiologic data, and cognition state datavia Task questions and answers and the subject's behavioral-analyzinginteractions with the user interface (204). After analyzing thecollected affective state sensory data and cognition state data, thecompliance state 205 is updated. The updated compliance state 205 iscompared with the target compliance level 201. The determinedknowledge/skill gap or the fit of the task procedural instructiondelivery strategy 206 is provided to the Training Model again,completing the loop. If the analysis on the collected affective statesensory data and cognition state data shows a probability of achievingcertain compliance level that is higher than a threshold, that certaincompliance level is considered achieved 207.

FIG. 3 depicts an activity diagram illustrating in more details theexecution process of the system for administering, evaluating, andmonitoring a subject's compliance with task performance requirementswithin an action programme under the Subject Model and Training Model.In an exemplary embodiment referring to FIG. 3, the execution process isas follows:

-   301. A subject logs into the system via her computing device running    a user interface rendered by the system's communication module.-   302. The subject select a Curriculum presented to her in the user    interface.-   303. Upon receiving the user login, successful authentication, and    receiving the Curriculum selection, the system's trainer module,    running in a server computer, selects and requests from the Domain    Knowledge repository one or more Task objects associated with the    Curriculum selected. When no Task object has yet been defined to    associate with any Concept objects in the Curriculum selected, the    system evaluates the Knowledge Tree and finds the Concept Knowledge    and Skills that the subject has not yet learned and/or been    evaluated as close to the root (fundamental) of the Knowledge Tree    as possible. This process is executed by the system's recommendation    engine, which can be implemented by a combination of software and    firmware executed in general purposed and specially designed    computer processors. The recommendation engine can recommend    Practice Tasks, and at lower rate Mastery Challenge Tasks.    System-recommended Tasks have a default priority; compliance    officer-assigned Tasks have a higher priority in the Task selection.    In one embodiment, the system further comprises a recommendation    engine for recommending the task performance specification materials    (e.g. topic) to be learned next in a Curriculum. Using the estimated    affective state and cognitive state data of the subject, performance    data of the subject, the Knowledge Tree (with all ‘edge’ topics    listed), the compliance officer's recommendation information, data    from collaborative filters (look at data from peer subjects), and    task performance specification content data (match subject    attributes with the task performance specification material's    attributes), the recommendation engine recommends the next Task to    be executed by the system under the Training Model. For example, the    subject's negative emotion can be eased by recognizing the    difficult/unfamiliar topics (from the affective state data estimated    during the execution of certain Task) and recommending the next Task    of a different/more familiar topic; and recommending the next Task    of a difficult/unfamiliar topic when subject's emotion state is    detected position. In another example, the recommendation engine can    select the next Task of higher difficulty when the estimated    affective state data shows that the subject is unchallenged. This    allows the matching of Tasks with the highest compliance level    gains. This allows the clustering of Tasks based on similar    performance data and/or affective state and cognitive state    estimation. This also allows the matching of subject peers with    similar compliance level accomplishment to form focus groups.-   304. If the requested Task objects are found, their data are    retrieved and are sent to the subject's computing device for    presentation in the system's communication module user interface.-   305. The subject selects a Task object to begin the compliance    evaluation and monitoring session.-   306. The system's trainer module retrieves from the Domain Knowledge    repository the next item in the selected Task object for rendering    in the system's communication module user interface.-   307. Entering the Subject Model, the system's communication module    user interface renders the item (compliance questionnaire question    and/or task procedure instruction) in the selected Task object.-   308. A camera for capturing the subject's face is activated.-   309. During the subject's engagement in task procedure materials in    the item in the selected Task object (309 a), the subject's    point-of-gaze and facial expressions are analyzed (309 b).-   310. Depending on the estimated affective state and cognitive state    of the subject based on sensory data collected and information in    the subject's profile (overlay, includes all past performance data    and compliance level achievement progress data), virtual assistant    may be presented in the form of guidance and/or textual hint    displayed in the system's communication module user interface.-   311. The subject submits an attempt answer and/or an attempt command    for completing a task procedural step.-   312. The attempt answer and/or attempt command is graded and the    grade is displayed to the subject in the system's communication    module user interface.-   313. The attempt answer and/or attempt command and grade is also    stored by the system for further analysis.-   314. The attempt answer and/or attempt command and grade are used in    calculating the probability of the subject's understanding of the    Concept associated with the selected Task object and the probability    of the subject achieving a target compliance level in the task.-   315. If the selected Task is completed, the system's trainer module    selects and requests the next Task based on the calculated    probability of the subject's understanding of the associated Concept    and the probability of the subject achieving a target compliance    level in the task, and repeat the steps from step 303.-   316. If the selected Task is not yet completed, the system's trainer    module retrieves the next item in the selected Task and repeat the    steps from step 306.-   317. After all Tasks are completed, the system generates the result    report for subject.

In accordance to another aspect of the present invention, the system foradministering, evaluating, and monitoring a subject's compliance withtask performance requirements within an action programme furthercomprises an administration module that takes information from thecompliance officers, subjects, and Domain Knowledge in offeringassistance with the operation of face-to-face compliance evaluation andmonitoring process across multiple physical facilities as well asonline, remote evaluation and monitoring. In an exemplary embodiment,the administration module comprises a constraint-based schedulingalgorithm that determines the optimal scheduling of complianceevaluation and monitoring sessions while observing constraints suchcompliance officers' certification, travelling distance for subjects andcompliance officers, first-come-first-served, composition of thecompliance officers group based on compliance level achievement progressand training strategy. For example, when the compliance officer wants topromote peer teaching/training, the scheduling algorithm can selectsubjects with complementary skill sets so that they can help each otherand form focus groups.

An in-person face-to-face compliance evaluation and monitoring sessionmay comprise a typical flow such as: subjects check in, perform a smalltask to evaluate the cognitive state of the subjects, and the resultsare presented on the compliance officer's user interface dashboarddirectly after completion. The session then continues with explanationof a new task performance specification by the compliance officer, herethe compliance officer receives assistance from the system's pedagogicalagent with pedagogical goals and hints. After the explanation, thesubjects may engage in the new task in which the system provides as muchscaffolding as needed. Based on the compliance level achievementprogress and affective states of the subjects, the system's trainermodule decides how to continue the compliance evaluation and monitoringsession with a few options: e.g. recommend to form focus groups eachwith subjects who have achieved similar compliance levels in prior taskscompleted. The compliance evaluation and monitoring session is concludedby checking out. The attendance data is collected for billing purposesand for compliance certification purposes.

Although the embodiments of the present invention described above areprimarily applied in commercial and industrial activities, surveying,and job performance assessment settings, the present invention can beadapted without undue experimentation to customer relationshipmanagement (CRM) action programmes. In accordance to one embodiment ofthe present invention, the method and system for administering,evaluating, and monitoring a subject's compliance with task performancerequirements within an action programme comprise a mechanism fordelivering and managing interactive and adaptive compliancequestionnaire and task procedural instructions. The mechanism logicallystructures compliance questionnaire and task procedural instructionmaterials and the delivery mechanism data in a compliance programme as aDomain Knowledge, with its constituent Concept objects and Task objectshaving Knowledge and Skill items, and training materials respectivelythat are relevant to the concerned industry or trade. In the applicationof surveying, the system's estimation of the subjects' affective statesand cognitive states can be used in driving the selection andpresentment of survey questions. This in turn enables more accurate andspeedy survey results procurements from the subjects. In the applicationof job performance assessment, the system's estimation of the employeesubjects' affective states and cognitive states on duty continuouslyallows an employer to gauge the skill levels, engagement levels, andinterests of the employees and in turn provides assistance in work androle assignments.

Referring to FIGS. 6A and 6B. In accordance to another aspect of thepresent invention, the method and system for method and system foradministering, evaluating, and monitoring a subject's compliance withtask performance requirements within an action programme incorporatemachine learning techniques that are based on interlinking models ofexecution comprising: a Domain Model, an Assessment Model, a LearnerModel, a Deep Learner Model, one or more Motivational Models, aTransition Model, and a Pedagogical Model. The interlinking models ofexecution is purposed for driving, inducing, or motivating certaindesirable actions, behavior, and/or outcome from the subject. Thesecertain desirable actions and/or outcome can be, as non-limitingexamples, learning certain subject matters, achieving certain academicgoals, achieving certain career goals, completing certain jobassignments, making certain purchases, and conducting certain commercialactivities. These interlinking models of execution together form amachine learning feedback loop comprising the continuous tracking andassessment of learning progress of the subject under the AssessmentModel, driving the learning activities under the Learner Model,motivating the subject under the Deep Learner Model and MotivationOperational Model, and selecting and re-selecting knowledge space itemsunder the Domain Model and Transition Model, and delivering theknowledge space items and activities from one knowledge state to thenext under the Pedagogical Model.

The electronic embodiments disclosed herein may be implemented usinggeneral purpose or specialized computing devices, computer processors,or electronic circuitries including but not limited to applicationspecific integrated circuits (ASIC), field programmable gate arrays(FPGA), and other programmable logic devices configured or programmedaccording to the teachings of the present disclosure. Computerinstructions or software codes running in the general purpose orspecialized computing devices, computer processors, or programmablelogic devices can readily be prepared by practitioners skilled in thesoftware or electronic art based on the teachings of the presentdisclosure.

All or portions of the electronic embodiments may be executed in one ormore general purpose or computing devices including server computers,personal computers, laptop computers, mobile computing devices such assmartphones and tablet computers.

The electronic embodiments include computer storage media havingcomputer instructions or software codes stored therein which can be usedto program computers or microprocessors to perform any of the processesof the present invention. The storage media can include, but are notlimited to, floppy disks, optical discs, Blu-ray Disc, DVD, CD-ROMs, andmagneto-optical disks, ROMs, RAMs, flash memory devices, or any type ofmedia or devices suitable for storing instructions, codes, and/or data.

Various embodiments of the present invention also may be implemented indistributed computing environments and/or Cloud computing environments,wherein the whole or portions of machine instructions are executed indistributed fashion by one or more processing devices interconnected bya communication network, such as an intranet, Wide Area Network (WAN),Local Area Network (LAN), the Internet, and other forms of datatransmission medium.

The foregoing description of the present invention has been provided forthe purposes of illustration and description. It is not intended to beexhaustive or to limit the invention to the precise forms disclosed.Many modifications and variations will be apparent to the practitionerskilled in the art.

The embodiments were chosen and described in order to best explain theprinciples of the invention and its practical application, therebyenabling others skilled in the art to understand the invention forvarious embodiments and with various modifications that are suited tothe particular use contemplated.

What is claimed is:
 1. A system for administering, evaluating, andmonitoring a subject's compliance with task performance requirementswithin an action programme comprising: one or more optical sensorsconfigured for capturing and generating sensory data on a subject duringa compliance evaluation and monitoring session; one or more electronicdatabases including one or more domain knowledge data entities, eachdomain knowledge data entity comprising one or more concept dataentities and one or more task data entities, wherein each concept dataentity comprises one or more and task performance specification contentitems, wherein each task data entity comprises one or more compliancequestionnaire and task procedural instruction material items, whereineach task data entity is associated with at least one concept dataentity, and wherein a curriculum is formed by grouping a plurality ofthe concept data entities; a subject module executed by one or morecomputer processing devices configured to estimate the subject'saffective state and cognitive state using the sensory data collectedfrom the optical sensors; a trainer module executed by one or morecomputer processing devices configured to select a subsequent task dataentity and retrieve from the electronic databases the task data entity'scompliance questionnaire and task procedural instruction material itemsfor delivery and presentment to the subject after each completion of atask data entity in the compliance evaluation and monitoring session;and a recommendation engine executed by one or more computer processingdevices configured to create a list of task data entities available forselection of the subsequent task data entity, wherein the task dataentities available for selection are the task data entities associatedwith the one or more concept data entities forming the curriculumselected; wherein the selection of a task data entity from the list oftask data entities available for selection is based on a probability ofthe subject achieving a target compliance level in a task associatedwith the concept data entity's task performance specification contentitems; and wherein the probability of the subject achieving the targetcompliance level is computed using input data of the estimation of thesubject's affective state and cognitive state.
 2. The system of claim 1,further comprising: one or more physiologic measuring devices configuredfor capturing one or more of the subject's tactile pressure exerted on atactile sensing device, heart rate, electro dermal activity (EDA), skintemperature, and touch response, and generating additional sensory dataduring the compliance evaluation and monitoring session; wherein thesubject module is further configured to estimate the subject's affectivestate and cognitive state using the sensory data collected from theoptical sensors and the additional sensory data collected from thephysiologic measuring devices.
 3. The system of claim 1, furthercomprising: one or more voice recording devices configured for capturingthe subject's voice and speech clarity, and generating additionalsensory data during the compliance evaluation and monitoring session;wherein the subject module is further configured to estimate thesubject's affective state and cognitive state using the sensory datacollected from the optical sensors and the additional sensory datacollected from the voice recording devices.
 4. The system of claim 1,further comprising: one or more handwriting capturing devices configuredfor capturing the subject's handwriting, and generating additionalsensory data during the compliance evaluation and monitoring session;wherein the module is further configured to estimate the subject'saffective state and cognitive state using the sensory data collectedfrom the optical sensors and the additional sensory data collected fromthe handwriting capturing devices.
 5. The system of claim 1, furthercomprising: one or more pedagogical agents configured for capturing thesubject's interaction with the pedagogical agents, and generatingadditional sensory data during the compliance evaluation and monitoringsession; wherein the module is further configured to estimate thesubject's affective state and cognitive state using the sensory datacollected from the optical sensors and the additional sensory datacollected from the pedagogical agents.
 6. The system of claim 1, whereineach of the task performance specification content material items is anoperation note, an illustration, a test question, a video with anembedded test question, a problem-solving exercise having multiple stepsdesigned to provide guidance in deriving a solution to a problem, or aproblem-solving exercise having one or more heuristic rules orconstraints for simulating problem-solving exercise steps delivered insynchronous with the subject's performance of the task procedural stepsassociated with the task performance specification.
 7. The system ofclaim 1, wherein a plurality of the concept data entities are linked toform a logical tree data structure; wherein concept data entities havingknowledge and skill content items that are fundamental in a topic arerepresented by nodes closer to a root of the logical tree data structureand concept data entities having knowledge and skill content items thatare advance and branches of a common fundamental knowledge and skillcontent item are represented by nodes higher up in different branches ofthe logical tree data structure; wherein the recommendation engine isfurther configured to create a list of task data entities available forselection of the subsequent task data entity, wherein the task dataentities available for selection are the task data entities associatedwith the one or more concept data entities forming the curriculumselected and the one or more concept data entities having knowledge andskill items not yet mastered by the subject and as close to the roots ofthe logical tree data structures that the concept data entitiesbelonging to.
 8. The system of claim 1, wherein the probability of thesubject achieving the target compliance level is computed using inputdata of the estimation the subject's affective state and cognitive stateand the subject's performance data and behavioral data; and wherein thesubject's performance data and behavioral data comprises one or more ofnumber of successful and unsuccessful attempts to task procedural stepcompletions, speed in completing task procedures, correctness of answersto questions in the questionnaire, number of successful and unsuccessfulattempts to questions, closeness of the subject's answers to modelanswers, toggling between given answer choices, and response speed totest questions of certain types, subject matters, and/or taskperformance specification complexity/difficulty/stringency levels,working steps toward a solution, the subject's handwriting, tone ofvoice, and speech clarity.
 9. The system of claim 1, wherein the sensorydata comprises one or more of the subject's facial expression, eyemovements, point-of-gaze, and head pose.
 10. The system of claim 1,wherein the selection of a task data entity from the list of task dataentities available for selection is based on the probability of thesubject achieving the target compliance level and the subject'sestimated affective state; wherein when the subject's estimatedaffective state indicates a negative emotion, a task data entity that isassociated with a concept data entity having knowledge and skill contentitems that are favored by the subject is selected over another task dataentity that is associated with another concept data entity havingknowledge and skill content items that are disliked by the subject; andwherein when the subject's estimated affective state indicates apositive emotion, a task data entity that is associated with a conceptdata entity having knowledge and skill content items that are dislikedby the subject is selected over another task data entity that isassociated with another concept data entity having knowledge and skillcontent items that are favored by the subject.
 11. A method foradministering, evaluating, and monitoring a subject's compliance withtask performance requirements within an action programme comprising:capturing and generating sensory data on a subject using one or moreoptical sensors during the compliance evaluation and monitoring session;providing one or more electronic databases including one or more domainknowledge data entities, each domain knowledge data entity comprisingone or more concept data entities and one or more task data entities,wherein each concept data entity comprises one or more task performancespecification content items, wherein each task data entity comprises oneor more compliance questionnaire and task procedural instructionmaterial items, wherein each task data entity is associated with atleast one concept data entity, and wherein a curriculum is formed bygrouping a plurality of the concept data entities; estimating thesubject's affective state and cognitive state using the sensory datacollected from the optical sensors; and selecting a subsequent task dataentity and retrieving from the electronic databases the task dataentity's compliance questionnaire and task procedural instructionmaterial items for delivery and presentment to the subject after eachcompletion of a task data entity in the compliance evaluation andmonitoring session; creating a list of task data entities available forselection of the subsequent task data entity, wherein the task dataentities available for selection are the task data entities associatedwith the one or more concept data entities forming the curriculumselected; wherein the selection of a task data entity from the list oftask data entities available for selection is based on a probability ofthe subject achieving a target compliance level in a task associatedwith the concept data entity's task performance specification contentitems; and wherein the probability of the subject achieving the targetcompliance is computed using input data of the estimation of thesubject's affective state and cognitive state.
 12. The method of claim11, further comprising: capturing and generating additional sensory dataon one or more of the subject's tactile pressure exerted on a tactilesensing device, heart rate, electro dermal activity (EDA), skintemperature, and touch response during the compliance evaluation andmonitoring session; wherein the estimation of the subject's affectivestate and cognitive state uses the sensory data collected from theoptical sensors and the additional sensory data.
 13. The method of claim11, further comprising: capturing and generating additional sensory dataon the subject's voice and speech clarity using one or more voicerecording devices during the compliance evaluation and monitoringsession; wherein the estimation of the subject's affective state andcognitive state uses the sensory data collected from the optical sensorsand the additional sensory data collected from the voice recordingdevices.
 14. The method of claim 11, further comprising: capturing andgenerating additional sensory data on the subject's handwriting usingone or more handwriting capturing devices during the complianceevaluation and monitoring session; wherein the estimation of thesubject's affective state and cognitive state uses the sensory datacollected from the optical sensors and the additional sensory datacollected from the handwriting capturing devices.
 15. The method ofclaim 11, further comprising: capturing and generating additionalsensory data on the subject's interaction with one or more pedagogicalagents during the compliance evaluation and monitoring session; whereinthe estimation of the subject's affective state and cognitive state usesthe sensory data collected from the optical sensors and the additionalsensory data collected from the pedagogical agents.
 16. The method ofclaim 11, wherein each of the task performance specification contentmaterial items is an operation note, an illustration, a test question, avideo with an embedded test question, a problem-solving exercise havingmultiple steps designed to provide guidance in deriving a solution to aproblem, or a problem-solving exercise having one or more heuristicrules or constraints for simulating problem-solving exercise stepsdelivered in synchronous with the subject's performance of the taskprocedural steps associated with the task performance specification. 17.The method of claim 11, wherein a plurality of the concept data entitiesare linked to form a logical tree data structure; wherein concept dataentities having knowledge and skill content items that are fundamentalin a topic are represented by nodes closer to a root of the logical treedata structure and concept data entities having knowledge and skillcontent items that are advance and branches of a common fundamentalknowledge and skill content item are represented by nodes higher up indifferent branches of the logical tree data structure; wherein the taskdata entities available for selection are the task data entitiesassociated with the one or more concept data entities forming thecurriculum selected and the one or more concept data entities havingknowledge and skill items not yet mastered by the subject and as closeto the roots of the logical tree data structures that the concept dataentities belonging to.
 18. The method of claim 11, wherein theprobability of the subject achieving a target compliance level iscomputed using input data of the estimation the subject's affectivestate and cognitive state and the subject's performance data andbehavioral data; and wherein the subject's performance data andbehavioral data comprises one or more of number of successful andunsuccessful attempts to task procedural step completions, speed incompleting task procedures, correctness of answers to questions in thequestionnaire, number of successful and unsuccessful attempts toquestions, closeness of the subject's answers to model answers, togglingbetween given answer choices, and response speed to test questions ofcertain types, subject matters, and/or task performance specificationcomplexity/difficulty/stringency levels, working steps toward asolution, the subject's handwriting, tone of voice, and speech clarity.19. The method of claim 11, wherein the sensory data comprises one ormore of the subject's facial expression, eye movements, point-of-gaze,and head pose.
 20. The method of claim 11, wherein the selection of atask data entity from the list of task data entities available forselection is based on the probability of the subject achieving thetarget compliance level and the subject's estimated affective state;wherein when the subject's estimated affective state indicates anegative emotion, a task data entity that is associated with a conceptdata entity having knowledge and skill content items that are favored bythe subject is selected over another task data entity that is associatedwith another concept data entity having knowledge and skill contentitems that are disliked by the subject; and wherein when the subject'sestimated affective state indicates a positive emotion, a task dataentity that is associated with a concept data entity having knowledgeand skill content items that are disliked by the subject is selectedover another task data entity that is associated with another conceptdata entity having knowledge and skill content items that are favored bythe subject.